Layered Control Architectures in Robots and Vertebrates

We review recent research in robotics, neuroscience, evolutionary neurobiology, and ethology with the aim of highlighting some points of agreement and convergence. Specifically, we compare Brooks’ (1986) subsumption architecture for robot control with research in neuroscience demonstrating layered control systems in vertebrate brains, and with research in ethology that emphasizes the decomposition of control into multiple, intertwined behavior systems. From this perspective we then describe interesting parallels between the subsumption architecture and the natural layered behavior system that determines defense reactions in the rat. We then consider the action selection problem for robots and vertebrates and argue that, in addition to subsumption-like conflict resolution mechanisms, the vertebrate nervous system employs specialized selection mechanisms located in a group of central brain structures termed the basal ganglia. We suggest that similar specialized switching mechanisms might be employed in layered robot control architectures to provide effective and flexible action selection. keywords: subsumption architecture, brain evolution, behavior systems, defense system, action selection, basal ganglia. citation: Prescott, T.J., Redgrave, P., & Gurney, K. (1999). Layered control architectures in robots and vertebrates, Adaptive Behavior, 7, 99-127. Prescott et al. Layered Control Architectures 2 Layered Control Architectures in Robots and Vertebrates The field of adaptive behavior seeks a convergence of ideas from the different disciplines that study artificial and natural autonomous systems. Demonstrating convergence allows the interchange of concepts and ideas and enriches our understanding of both the biological and the synthetic (Arbib, 1989; Meyer & Guillot, 1990). Within this tradition the present article reviews research in robotics, neuroscience, evolutionary neurobiology, and ethology, with the aim of highlighting some key areas of agreement, and argues that this cross-disciplinary perspective could help to resolve some of the current dilemmas facing research in autonomous robotics. Rodney Brooks’ (1986, 1989, 1990, 1991ab, 1995) work in engineering robot “creatures” needs little introduction to researchers in adaptive behavior. In the mid-eighties Brooks introduced a new methodology—based on an analogy with natural evolution—for building “self-sustaining” mobile robots that operate in real-time and in un-customized human environments. This research has had enormous influence in robotics and, together with other contemporary work that proposed a move towards more distributed and situated systems (e.g. Braitenberg, 1986; Minsky, 1986), has inspired a new research paradigm in artificial intelligence (see e.g. Meyer & Guillot, 1990; Maes, 1992). A key contribution of Brooks’ work is his proposal for a layered, distributed control architecture for mobile robots, termed the subsumption architecture (SA). Section 1 of this article briefly outlines the key features of the SA. A substantial body of the neuroscience literature can be interpreted as demonstrating layered control systems in the vertebrate brain. In many ways the notion of layering is a common, often unspoken, assumption in contemporary neuroscience, however, the implications of the layered nature of the brain are not always acknowledged in a field dominated by the study of the mammalian cortex. Section 2 considers work that follows in the tradition of John Hughlings Jackson (1884/1958), a neuropsychologist who is particularly associated with the notion of layered competence, while Section 3 looks for similarities between the robot design process proposed by Brooks and the evolutionary history of the vertebrate brain. An understanding of adaptive behavior is central to the behavior systems approach which stems from pioneering work in ethology by Lorenz, Tinbergen, and Baerends (see Baerends, 1976), and has been influential in some recent research in psychology and neuroscience (see Timberlake, 1993). A key principle is that the functional organization of the vertebrate brain can be decomposed into multiple, semi-independent, systems dedicated to major biological functions such as feeding, reproduction, defense, and body care. Section 4 gives a brief outline of the behavior systems approach relating it to Brooks' proposal for behavior-based robot control. A general thesis is best served by a specific example. In Section 5 we argue that the layered neural architecture that implements the defense behavior system in the rat bears many Prescott et al. Layered Control Architectures 3 interesting resemblances to Brooks' SA. A number of specific correspondences are outlined in detail. Finally, in Section 6, we consider the action selection problem for both natural and artificial control systems, distinguishing between emergent and specialized action selection, and between distributed and centralized selection mechanisms. We then note that the basal ganglia (BG), a group of functionally-related, central brain structures, have a controlling influence on neural systems at multiple levels of the vertebrate nervous system and so form an important exception to the overall vertical decomposition of the brain. The BG, we propose, act as a specialized action selection device that provides flexible conflict resolution between functional units that are widely distributed in the brain. Although providing a centralized selection system within the global brain architecture, the BG exploits the advantages of distributed switching at a local level. Understanding the function of the BG within the vertebrate brain, we suggest, could help in the design of effective action selection mechanisms for robots, supplementing the use of layered, subsumption-style control. 1 BROOKSÕ SUBSUMPTION ARCHITECTURE Brooks (1986) introduced the subsumption architecture (SA) as the central element of his proposal for building “complete creatures” capable of sustained activity in everyday human environments. With only limited modification (Brooks, 1989) the principles of the SA have since been employed in the design and control of a large number of mobile robots (see Brooks, 1990, 1991b) and have been widely copied. The key aspects of the SA, of most relevance here, are as follows: Distributed, layered control. Control in a Brooks’ robot is distributed across several layers each composed of multiple modules often mounted on different processors. Layers operate in parallel and asynchronously. Within a layer there is no central control module. Behavioral decomposition. The different layers of the control system are designed to support different “task-achieving” behaviors (such as obstacle avoidance, wandering, exploring, map-building); the problem of controlling the robot is thus decomposed into behavioral units rather than into different “functional” units (such as perception, modeling, planning, and motor control). Within a layer there may be a more traditional decomposition; for instance, into sensor and actuator components. However, different layers will use different decompositions based on specialized sub-sets of the available sensorimotor apparatus. Increasing “levels of competence.” Each ascending level of the control system adds to the behavioral capabilities of the robot resulting in a higher level of overall competence. Damage or failure at a higher level reduces the robot to functioning at the level of the next highest layer. The higher layers of the SA often operate by modulating the activity of lower layers—hence their contact with the motor resources can be relatively indirect. Since higher level modules can implicitly rely on the operation of lower-level behavioral primitives, they can be designed to generate more complex or subtle motor acts. Brooks' distinction between functional and behavioral components is not in common use in biology. Here the term function will generally be used to indicate the purpose or use of a mechanism as opposed to its form or structure. Where the distinction that Brooks has proposed is to be considered this will be made clear in the text. Prescott et al. Layered Control Architectures 4 Incremental construction. A key constraint on the design process is that, as each additional level of competence is incorporated, the total system should be “a strict augmentation of the previous one” (Brooks, 1989 p. 253). Designing the control system is therefore an incremental process in which each intermediate architecture is extensively tested and debugged before the next layer is added. Conflict resolution and communication between levels by subsumption mechanisms. Higher layers of the control system can subsume the roles of lower ones by suppressing their outputs and (optionally) substituting their own. Each lower level continues to function as higher levels are added, “unaware” that those above may be interfering with its data paths. Little sensor fusion and no central models. Brooks’ (1986) places less emphasis on the combination of multiple sensor signals to determine the most accurate estimate possible of world state (sensor fusion) than on the independent use of different sources of sense data to provide robustness to changing environmental conditions, sensor noise, or hardware failure (see also Prescott, 1996). One consequence of this view is the principle that the robot should have no need for central world models into which all available sense data is compiled. Rather than exploiting shared representations, behaviors at different levels are separated by “abstraction barriers” (Brooks, 1990), unable to influence each other’s internal workings by anything more than simple subsumption mechanisms. Internal state at each higher level cannot be accessed by lower layers, although higher layers can access the data paths of those below. The Subsumption Metaphor in Biology Though the main impact of Brooks’ work has clearly been in robotics and AI, it has also had a significant influence on the study of natural intelligence. Work in this wide area that acknowledges the influence of Brooks’ approach includes studies of human perception and motor control (Ballard, Hayhoe, & Whitehead, 1992; King, Dykeman, Redgrave et al., 1992), human development (Rutkowska, 1994), and research in computational neuroethology (e.g. Altman & Kien, 1989; Arbib & Liaw, 1995; Cliff, 1991; Franceschini, 1992). Brooks himself does not make strong claims for the SA as a model for understanding natural autonomous systems. Indeed, he explicitly states that, although the SA draws on an evolutionary metaphor, it is not a biological model. He also warns of the dangers of treating biological intelligence as a lodestar for AI (Brooks, 1995). However, Brooks also insists that his interest is in general intelligence (Brooks, 1995), that he sees animal intelligence as an important “existence proof of the possibility of intelligent entities” (Brooks, 1990, p. 5), and that we should expect to gain insights for robot design by studying the nervous systems and behavior of animals (Brooks, 1991a; Brooks, 1995). The search for further links between Brooks’ robot architectures and our understanding of animal intelligence therefore fits naturally with the situated robotics approach. The SA is not unique, of course, in being a layered robot control architecture. For example, Arbib and Liaw (1995), Roitblat (1991), McGonigle (1990), and Albus (1991), have each proposed architectures for robot control inspired by biological intelligence and sharing a number of interesting similarities with the architecture of the vertebrate brain. Several architectures have also been proposed that are principally refinements or extensions of the SA (e.g. Rosenblatt & Payton, 1989; Connell, 1990). This article focuses on the original SA, Prescott et al. Layered Control Architectures 5 however, as it is probably the best known and most imitated architecture in behavior-based robotics. Drawing comparisons with such a widely understood model will, we hope, encourage robot designers to look with greater interest at the organization of the vertebrate nervous system as a source of inspiration for the design of robot control architectures. 2 THE VERTEBRATE BRAIN VIEWED AS A LAYERED ARCHITECTURE The Jacksonian Perspective in Neuroscience In 1884, in a famous lecture on the “evolution and dissolution of the nervous system” the neurologist John Hughlings Jackson (1884/1958) proposed a layered view of the nervous system, in which the brain is seen as implementing multiple levels of sensorimotor competence. Jackson’s view, inspired by the Darwinian revolution in nineteenth century science, was based not on the usual morphological divisions but on functional grounds, “ a s to the degree of indirectness with which each [division] represents the body, or part of it” (p. 53). He divided the nervous system into lower, middle, and higher centers, and proposed that this sequence represented a progression from the “most organized” (most fixed) to the “least organized” (most modifiable), from the “most automatic” to the “least automatic,” and from the most “perfectly reflex” to the least “perfectly reflex.” This progression sees an increase in competence in a manner that we might now understand as a behavioral decomposition—higher centers are concerned with same sort of sensorimotor coordinations as those below, though in a more indirect fashion: That the middle motor centers represent over again what all the lowest motor centers have represented, will be disputed by few. I go further, and say that the highest motor centers (frontal lobes) represent over again, in more complex combinations, what the middle motor centers represent. In recapitulation, there is increasing complexity, or greater intricacy of representation, so that ultimately the highest motor centers represent, or, in other words, coordinate, movements of all parts of the body in the most special and complex combinations. (Jackson 1884/1958 p. 53) Jackson viewed the evolution of the nervous system as an incremental process in which lower levels are retained intact but are suppressed by higher systems. Within the different centers Jackson further considered there to be functionally distinct layers. He argued for a dissociation of higher layers from those below such that a breakdown at a higher layer—a “dissolution” in Jackson’s terminology—caused a reversion to the next highest layer of control. There are further important parallels between Jackson’s writing and contemporary approaches in robotics. For instance, he was an early of advocate of the notion of distributed representation. Overall, his writings show a conviction that “higher” thought is grounded in A number of Hughlings Jackson's contemporaries held similar views on brain organization, for review see Magoun (1958), Berntson, Boysen and Cacioppo (1993). Prescott et al. Layered Control Architectures 6 perception and action—a perspective which, while radical for his era, is clearly in sympathy with recent proponents of situated action (e.g. Brooks, 1990, 1995; Chapman, 1991): A man physically regarded is a sensorimotor mechanism. I particularly wish to insist that the highest centers—physical basis of mind or consciousness—have this kind of constitution, that they represent innumerable different impressions and movements of all parts of the body [..] It may be rejoined that the highest centers are “for mind.” I assert that they are “for body,” too. If the doctrine of evolution be true, all nervous centers must be of sensorimotor constitution. (Jackson, 1884/1958, p. 63) Jackson’s views on the functional organization of the nervous system continue to influence and inspire neuroscientific research (see e.g. Teitelbaum, Schallert & Whishaw, 1983; Rudy, Stadler-Morris & Albert, 1987; King et al., 1992; Berntson, Boysen & Cacioppo, 1993), and there is now a mass of empirical evidence—anatomical, physiological and behavioral—that supports the notion of layered control systems in the vertebrate brain. Some of the experimental support for this view is briefly summarized below.